New BUILD Multiplayer Code Specification

by Jonathon Fowler (jonof@edgenetwk.com)


Goals
-----

* To have reliable packet delivery.
* To have high interactivity.
* To keep retransmission at a practical minimum.


Packet Structure
----------------

unsigned long  crc;			// crc32 of the entire packet data (including header, but excluding this property)
unsigned short packetlength;		// total length of data this packet ought to contain (including this field)
unsigned char  messagecount;		// number of messages contained in this packet
{
unsigned char  sequence;		// sequence number of the message
unsigned char  length;			// the length of the message minus 1
unsigned char  flags;			// bits 0 & 1 represent message type
					//   00 = represents a synchronous message.
					//   01 = represents a message which doesn't affect sync.
					//   10 = represents a sysex/oob message which uses the sysex message format.
					//   11 = undefined
unsigned char  messagedata[length+1];	// data for this message
} * messagecount;			// sorted ascending by sequence number
unsigned char  acklen;			// byte-length of unacknowledged list
unsigned char  ack[];			// acknowledged sequence numbers, array size being calculated
					//   from (packetlength - headersize - messages - 1(acklen))

The acknowledgements table contains all the unacknowledged messages, followed by the list of messages we're
acknowledging as well.

The acknowledgements tables are compressed using the following scheme:
	The first byte in a table represents the first message in a sequence of messages.
	Each byte thereafter up to the length of the table represents the (un)acknowledged state of a message where one
	  bit corresponds to one message in the sequence.
	If encoding unacknowledged messages, the first byte is the first unacked message and a '1' bit in the following
	  sequence represents an unacked message, '0' being acked.
	If encoding acknowledged messages, the first byte is the first message being acknowledged and a '1' bit in the
	  following sequence represents the next acked message.

So, in a sequence like:
	*,*,*,*,f9,*,*,fc,fd,*,ff,00,01,02,*,*,05,06,07,*,*

The encoded representation would be:
	0xf9, 0x37, 0x9c

	0x13 being 11101100, or the next 8 bits after 0xf9.
	0x06 being 00111001, or the next 8 bits following.


1. A typical packet will contain the current data plus the data of a number of frames that preceeded it to help with
   preventing casual packet loss, but only if the preceeding messages have not been acknowledged yet. Fast networks may
   therefore require only one or two messages per packet.
2. A grace period is given per message for acknowledgement in case the occasional packet is delivered out of order or takes
   unusually long to make the journey and be acknowledged, but, if a packet with a sequence number unusually greater than the
   next expected acknowledged number is acknowledged, the packets between the expected number and the acknowledged number are
   transmitted again in the next packet. The grace period could be tuned for different networks based on typical lag and
   whatnot.


What If...
----------

...a single host disappears and messages fail to reach them?

The network code should retry transmission until a message reaches some arbitratry age after which the player is dropped.
The game action will halt in this case because the other players will not be getting the packets needed to continue playing
that tic.





 - NEW NETWORK METHODS!  TRY MY GAME IN MULTIPLAYER MODE!  MODEM!

   SUMMARY:  Lag Time: ZERO!  That's right Z.E.R.O. (0)
   Smoothness: Perfect on ALL computers!

      My new network code builds upon everything you've already
   programed with faketimerhandler, getpackets, movethings, and
   domovethings, so you don't need to throw away any of that
   great stuff.  There are actually 2 totally separate things I
   did to improve the multiplayer performance.  First I added a
   new network mode which sends packets in a different way (this
   is way Doom actually does it)  Then I found a way to reduce
   lag-time to 0 with perfect smoothness.  (Doom does NOT do
   this, but G&S like it anyway for some reason.)  I will first
   describe the new network mode:

   -------------------------------------------------------------
   1.   First, let me describe the new network mode.  Instead of
   a master/slave system, every computer sends its own controls
   to every other.  You are currently using the master/slave
   system which sends 2(n-1) packets per frame where n is the
   number of players.  My new network mode sends n(n-1) packets
   per frame and treats every player evenly.  See the chart
   below of packets per frame:

               2(n-1) method:   n(n-1) method:
   2 players        2             2
   3 players        4             6
   4 players        6            12
   5 players        8            20 (OUCH!)
   6 players       10            30 (OUCH!)
   7 players       12            42 (OUCH!)
   8 players       14            56 (OUCH!)

      You may be asking why I am bothering you with this new
   network method if it sends more packets then the old one?
   I'll explain:  With the old network method, slaves had to
   wait for their packets to take 2 trips before they could move,
   whereas with the new method the packets need to take only 1
   trip.  Also with the new method the players are treated
   evenly.  For 2 players, the new network method is definitly
   the mode of choice since it sends the same number of packets
   AND the packets can be smaller since each computer only needs
   to send its own controls to the other computer, not everyone's
   controls (good for modem play).  It's up to you what your
   break even point is before you switch into the old network
   method.  I recommend: 1-4 New, 5+ Old.
      Now let me explain how the new method REALLY works.  Since
   every computer must call the movement code in the same order
   to stay in sync, all computers must wait until every packet
   is received for that frame before it can actually go into
   the movement code.  You could say that all the computers are
   half-slaves.  Your computer should always be ahead of the
   other computers.  If you are player 0, the packets you
   currently have might look like this:

   Chart for Player 0:

             Player 0   Player 1   Player 2
   Tic 0:    GOTMINE------GOT--------GOT------>     MOVE!
   Tic 1:    GOTMINE------GOT--------GOT------>     MOVE!
   Tic 2:    GOTMINE--X WAITING...   GOT         CAN'T MOVE!
   Tic 3:    GOTMINE    WAITING... WAITING...    CAN'T MOVE!
   Tic 4:    GOTMINE    WAITING... WAITING...    CAN'T MOVE!

      As soon as player 0 receives player 1's next packet,
   player 0 can call domovethings for tic 2.
      One interesting complication of the new network method is
   the timing.  If for some reason player 0 sends packets faster
   than player 1, then player 1 will have no lag and player 0
   will start with twice the normal lag which will increase
   until bad things happen.  See this chart:

              Player 0's side:         |   Player 1's side:
            Player 0:  Player 1:       | Player 0:  Player 1:
   Tic 5:   GOTMINE      GOT    (MOVE) |   GOT      GOTMINE  (MOVE)
   Tic 6:   GOTMINE    WAITING         |   GOT      GOTMINE  (MOVE)
   Tic 7:   GOTMINE    WAITING         |   GOT      GOTMINE  (MOVE)
   Tic 8:   GOTMINE    WAITING         |   GOT      GOTMINE  (MOVE)
   Tic 9:   GOTMINE    WAITING         |   GOT      GOTMINE  (MOVE)
                                       |   GOT
                                       |   GOT
                                       |   GOT

      This can be corrected by sending a byte which tells the
   other computer how many packets behind it is.  You want the
   other computer to be a little behind.   Player 0's packet
   delta in this case would be 4.  Player 1's would be -3.

      Another interesting thing about this network method is
   that a slave's packet cannot be skipped like in the
   master/slave system.

